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Abstract 

Forecasting technological progress is of great interest to engineers, policy makers, and private investors. 
Several models have been proposed for predicting technological improvement, but how well do these 
models perform? An early hypothesis made by Theodore Wright in 1936 is that cost decreases as a power 
law of cumulative production. An alternative hypothesis is Moore's law, which can be generalized to 
say that technologies improve exponentially with time. Other alternatives were proposed by Goddard, 
Sinclair et al., and Nordhaus. These hypotheses have not previously been rigorously tested. Using 
a new database on the cost and production of 62 different technologies, which is the most expansive 
of its kind, we test the ability of six different postulated laws to predict future costs. Our approach 
involves hindcasting and developing a statistical model to rank the performance of the postulated laws. 
Wright's law produces the best forecasts, but Moore's law is not far behind. We discover a previously 
unobserved regularity that production tends to increase exponentially. A combination of an exponential 
decrease in cost and an exponential increase in production would make Moore's law and Wright's law 
indistinguishable, as originally pointed out by Sahal. We show for the first time that these regularities 
are observed in data to such a degree that the performance of these two laws is nearly tied. Our results 
show that technological progress is forecastable, with the square root of the logarithmic error growing 
linearly with the forecasting horizon at a typical rate of 2.5% per year. These results have implications 
for theories of technological change, and assessments of candidate technologies and policies for climate 
change mitigation. 

Introduction 

Innovation is by definition new and unexpected, and might therefore seem inherently unpredictable. But 
if there is a degree of predictability in technological innovation, understanding it could have profound 
implications. Such knowledge could result in better theories of economic growth, and enable more effective 
strategies for engineering design, public policy design, and private investment. In the area of climate 
change mitigation, the estimated cost of achieving a given greenhouse gas concentration stabilization 
target is highly sensitive to assumptions about future technological progress W. 

There are many hypotheses about technological progress, but are they any good? Which, if any, 
hypothesis provides good forecasts? In this paper, we present the first statistically rigorous comparison 
of competing proposals. 

When we think about progress in technologies, the first product that comes to mind for many is 
a computer, or more generally, information technologies. The following quote by Bill Gates captures 
a commonly held view: "Exponential improvement - that is rare - we've all been spoiled and deeply 
confused by the IT model" (2). But as we demonstrate here, information technologies are not special 
in terms of the functional form that describes their improvement over time. Information technologies 
show rapid rates of improvement, but many technologies show exponential improvement. In fact, all the 
technologies we study here behave roughly similarly: Information technologies closely follow patterns of 
improvement originally postulated by Wright for airplanes [3}j8^ , and technologies such as beer production 
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or offshore gas pipelines follow Moore's law [9 10 , but with a slower rate of improvement [8 11-15 . 

It is not possible to quantify the performance of a technology with a single number [16] . A computer, 
for example, is characterized by speed, storage capacity, size and cost, as well as other intangible charac- 
teristics such as aesthetics. One automobile may be faster while another is less expensive. For this study 
we focus on one common measure of performance: the inflation-adjusted cost of one "unit". This metric 
is suitable in that it can be used to describe many different technologies. However, the nature of a unit 
may change over time. For example, a transistor in a modern integrated circuit today may have quite 
different performance characteristics than its discrete counterpart in the past. Furthermore, the degree 



to which cost is emphasized over other performance measures may change with time 17 . We nonetheless 
use the changes in the unit cost as our measure of progress, in order to compare competing models using 
a sizable dataset. The crudeness of this approach only increases the difficulty of forecasting and makes 
it particularly surprising that we nonetheless observe common trends. 

Analysis 

We test six different hypotheses that have appeared in the literature (3,9,18 20 , corresponding to the 
following six functional forms: 

(1) 



b + n(t) 



The dependent variable y t is the unit cost of the technology measured in inflation adjusted dollars. The 
independent variables are the time t (measured in years), the annual production q t , and the cumulative 
production xt — Y)j—i Qi- The noise term n(t), the constants a, b and c and the predictor variables differ 
for each hypothesis. 

Moore 's law here refers to the generalized statement that the cost y of a given technology decreases 
exponentially with time, i.e. 

y t = Bexp(-mt), (2) 
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where m > and B > are constants [9]]12]. (We assume throughout that t > 0, and we have renamed 
a = —m and b — \ogB in Eq. 0). Moore's law postulates that technological progress is inexorable, i.e. 
it depends on time rather than controllable factors such as research and development. 

Wright's law, in contrast, postulates that cost decreases at a rate that depends on cumulative pro- 
duction, i.e. 

Vt = Bxt w , (3) 

where w > and B > are constants, and we have renamed a = —w and b — \ogB in Eq. 0. Wright's 
law is often interpreted to imply "learning by doing" [5][2l] . The basic idea is that cumulative production 
is a proxy for level of effort, so that the more we make the more we learn, and knowledge accumulates 
without loss. 



Another hypothesis is due to Goddard 18 , who argues that progress is driven purely by economies 
of scale, and postulates that: 

Vt = Bq^ s , (4) 
where s > and B > are constants, and we have renamed a — —s and b = logB in Eq. ((lj). 
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We also consider the three multi- variable hypotheses in Eq. Q: Nordhaus 20 combines Wright's law 
and Moore's lawQ and Sinclair, Klepper, and Cohen (SKC) [l9] combine Wright's law and Goddard's law. 
For completeness we also test Wright's law lagged by one year. Note that these methods forecast different 
things: Moore's law forecasts the cost at a given time, Wright's law at a given cumulative production, 
and Goddard's law at a given annual production. 

We test these hypotheses on historical data consisting of 62 different technologies that can be broadly 
grouped into four categories: Chemical, Hardware, Energy, and OtheiQ The data are sampled at annual 
intervals with timespans ranging from 10 to 39 years. The choice of these particular technologies was 
driven by availability - we included all of the data that we could find to assemble the largest database of 
its kind. For a detailed description see the Supporting Information. 

To compare the performance of each hypothesis we use hindcasting, which is a form of cross-validation. 
We pretend to be at time i and make a forecast for time j using hypothesis (functional form) / 

and data set d, where j > i. The parameters for each functional form are fit using ordinary least squares 
based on all data prior to time i, and forecasts are made based on the resulting regression^] We score 
the quality of forecasts based on the logarithmic forecasting error 

efdij = log yf ] - log y { /' d ' l) (5) 

The quality of forecasts is examined for all datasets and all hypotheses (and visualized as a three- 
dimensional error mountain, as shown in the Supporting Information). For Wright's law an illustration 
of the growth of forecasting errors as a function of the forecasting horizon is given in Fig. ??. 

Developing a statistical model to compare the competing hypotheses is complicated by the fact that 
errors observed at longer horizons tend to be larger than those at shorter horizons, and errors are corre- 
lated across time and across functional forms. After comparing many different possibilities (as discussed 
in detail in the Supporting Information), we settled on the following approach. Based on a search of 
the family of power transformations, which is known for its ability to accommodate a range of variance 
structures, we take as a response the square root transformation of the logarithmic error. This response 
was chosen to maximize likelihood when modeled as a linear function of the hindcasting horizon = target 
— origin = j — i, using a linear mixed effects model. 

Specifically, we use the following functional form to model the response. 

rfdij = \e f dij I ' 5 = a f + a d + {fi f + b d )(j + e fdij , (6) 

where rfdij is the expected root error. The parameters a.f and j3f depend on the functional form and 
are called fixed effects because they are the same for all datasets. af is the intercept and j3f is the slope 
parameter. 

The parameters a d and b d depend on the dataset, and are called random effects because they are 
not fitted independently, but are instead treated as dataset-specific random fluctuations from the pooled 
data. The quantities a d and b d arc additive adjustments to the average intercept and slope parameters 
af and fit, respectively, to take into account the peculiarities of each dataset d. 

In order to avoid adding 62 a d parameters plus 62 b d parameters, we treated the ( ? d ) pair as a 

\b d J 

two-dimensional random vector having a bivariate normal distribution with mean f J and variance- 



^^Note that the conclusions presented do not work against Nordhaus' point about the difficulty in separating learning 
from exogenous sources of change 120] . 

2 All data can be found in the online Performance Curve Database at pcdb.santafe.edu. 

3 An alternative is to adjust the intercepts to match the last point, which produces better short term forecasts. For 
example, for Moore's law this corresponds to using a log random walk of the form log yt+i = log yt — (J. + n(t), where n(t) 
is an IID noise term. We have not done this here to be consistent with the way these hypotheses have been historically 
presented. The method we have used here also results in more stable errors. Our purpose here is not to propose an optimal 
forecasting method, but rather to compare existing hypotheses. 
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covariance matrix ( a % ] . This approach dramatically reduces the number of parameters. We 



parameterize the dataset-specific adjustments as random deviations from the average a ^ a cos * °^ 

only 3 additional parameters instead of 2 x 62 = 124. This parsimonious approach makes maximum 
likelihood estimation possible by keeping the number of parameters in check. 

Finally, we add an Sfdij random field term to take into account the deviations from the trend. This 

is assumed to be a Gaussian stochastic process independent of the [ , rf ) random vector, having mean 0, 



and given ad and bd, having variance equal to a positive a 2 times the fitted values: 

Va,r(e fdij\a d ,b d ) = a 2 E (r fdij \ a d , b d ) (7) 

We also define an exponential correlation structure within each error mountain (corresponding to each 
combination of dataset and hypothesis, see the Supporting Information), as a function of the differences of 
the two time coordinates with a positive range parameter p and another small positive nugget parameter 
t] quantifying the extent of these correlations: 

Corr(£ /dii , Ef> d >i>j<) = 6 f f>5 dd >(l-r))exp{-(\i-i'\ + \j-j'\)/p}, (8) 

where the two Kronecker S functions ensure that each error mountain is treated as a separate entity. 

Equations ^ and ^ were chosen to deal with the observed heteroscedasticity (increasing variance 
with increasing logarithmic forecasting error) and the serial correlations along the time coordinates i 
(hindcasting origin) and j (hindcasting target). Based on the likelihood, an exponential correlation 
function provided the best fit. Note that instead of a Euclidean distance (root sum of the squares of 
differences), the Manhattan measure was used (the sum of the absolute differences), because it provided 
a better fit in terms of the likelihood. 

Using this statistical model, we compared five different hypotheses. (We removed the Nordhaus 
model from the sample because of poor forecasting performance. This model gave good in-sample fits 
but generated large and inconsistent errors when predicting out-of-sample, a signature of over-fitting.) 
Rather than the 62 x 5 x 2 = 620 parameters needed to fit each of the 62 datasets separately for each 
of the five functional forms, there are only 16 free parameters: 5 x 2 — 10 parameters a/ and f3f, three 
parameters for the covariance matrix of the bivariate random vector (a d ,b d ), and three parameters for 
the variance and autocorrelation of the residuals e fdij ■ 



Results and Discussion 

We fit the error model to the 37, 745 different Tf d ij data points using the method of maximum likelihood. 
In Fig. [2] we plot the expected root error r jij = cif+(3f(j — i) for the five hypotheses as a function of the 
hindcasting horizon. While there are differences in the performance of these five hypotheses, they are not 
dramatic. The intercept is tightly clustered in a range 0.16 < aj < 0.19 and the slope 0.024 < (3f < 0.028. 
Thus all the hypotheses show a large initial error, followed by a growth in the root error of roughly 2.5% 
per yeaiQ 

The error model allows us to compare each hypothesis pairwise to determine whether it is possible to 
reject one in favor of another at statistically significant levels. The comparisons are based on the intercept 
and slope of the error model of Eq. (J6|. The parameter estimates are listed in Tables SI and S3 and 
the corresponding p-values in Tables S2 and S4. For example, at the 5% level, the intercept of Goddard 
is significantly higher than any of the others and the slope of SKC is significantly greater than that of 
Wright, lagged Wright and Goddard. With respect to slope, Moore is at the boundary of being rejected 



4 This is a central tendency for the pooled data. 
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in favor of Wright. Fig. [2] makes the basic pattern clear: Goddard does a poorer job of forecasting at 
short times, whereas SKC and to a lesser extent Moore do a poorer job at long times. 

We thus have the surprising result that most of the methods are quite similar in their performance. 
Although the difference is not large, the fact that we can eliminate Goddard for short term forecasts 
indicates that there is information in the cumulative production not contained in the annual production, 
and suggests that there is a learning effect in addition to economies of scale. But the fact that Goddard is 
not that much worse indicates that much of the predictability comes from annual production, suggesting 
that economies of scale are important. (In our database technologies rarely decrease significantly in annual 
production; examples of this would provide a better test of Goddard's theory.) We believe the SKC model 
performs worse at long times because it has an extra parameter, making it prone to overfitting. 

Although Moore performs slightly worse than Wright, given the clear difference in their economic 
interpretation, it is surprising that their performance is so similar. A simple explanation for Wright's law 
in terms of Moore's law was originally put forward by Sahal [ 22| . He noted that if cumulative production 
grows exponentialljj^J i.e. 

x t = Aexp(gt), (9) 

then eliminating t between Eqs. ^ and ^ results in Wright's law, Eq. with w = m/g. Indeed, when 
we look at production vs. time we find that in almost every case the cumulative production increases 
roughly exponentially with time. This is illustrated in Fig. [3l where we show three representative examples 
for production and cost plotted as a function of time. Fig. |3| also shows histograms of R 2 values for fitting 
g and m for the 62 datasets. The agreement with exponential behavior ranges from very good to rather 
poor, but of course these are short time series and some of them are very noisy. 

We test this in Fig. [I] by plotting the measured value of Wd against the derived value Wd — m/g for 
each data set d. The values cluster tightly along the identity line, indicating that Sahal's conjecture is 
correct. 

The differences in the data sets can be visualized by plotting ad and bd as shown in Fig. [5] All 
but one data set is inside the 95% confidence ellipsoid, indicating that the estimated distribution of 
ifld, bd) is consistent with the bivariate normal assumption. The intercepts vary in a range roughly 
—0.10 < ad < 0.17 and the slopes —0.018 < bd < 0.015. Thus the variation in the corresponding 
logarithmic forecasting error for the different datasets is comparable to the average error for all datasets 
(Fig. J5J) and about an order of magnitude larger than the difference between the hypothesized laws 

(Fig. g . 

To illustrate the practical usefulness of our approach we make a forecast of the cost of electricity for 
residential scale photovoltaic solar systems (PV). Fig. [6] shows the best forecast (solid line) as well as 
the expected error (dashed lines J] The expected cost in 2020 is 6 cents/kWh, with a range (3, 12), and 
in 2030 it is 2 cents, with a range (0.4, 11)Q The current cost of the cheapest alternative, coal-fired 
electricity, is roughly 5 cents/kWrj^J In contrast to PV, coal-fired electricity is not expected to decrease 



in cost, and will likely increase if there are future penalties for CO2 emissions 23 

The key postulate that we have made here is that the processes generating the costs of technologies 
through time are generic except for technology-specific differences in parameters. This hypothesis is 
powerful in allowing us to to view any given technology as being drawn from an ensemble. This means 
that we can pool data from different technologies to make better forecasts, and most importantly, make 



5 Note that if production grows exponentially, cumulative production also grows exponentially with the same exponent. 

6 Note that these are not confidence limits, but rather projected absolute log deviations from the best forecast, calculated 
from Eq. m using a Moore , /3 M 

oore' a Photovoltaics2> an d i>Photovoltaics2- The sharp drop in the one year forecast relative 
to the last observed data point comes from the fact that forecasts are based on the average trend line, and because this 
data series is particularly long. PV costs have risen recently due to increased material costs and other effects, but many 
industry experts expect this to be a short-lived aberration from the long-term cost trend. See footnote 4 and Section 5 of 
the Supporting Information. 

7 This does not include the additional cost of energy storage technologies. Note also that this is for residential scale PV. 
Industrial scale PV is typically about two-thirds the cost of electricity from residential scale systems. 

8 This is the wholesale cost at the plant (busbar), which may be most directly comparable to industrial scale PV. 
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error estimates. This is particularly useful in studying technology trends, where available data is limited. 
Of course we must add the usual caveats about making forecasts - as Neils Bohr reputedly said, prediction 
is very difficult, especially of the future. 

Our analysis reveals that decreasing costs and increasing production appear to be closely related, and 
that the hypotheses of Wright and Moore are more similar than they might appear. We should stress, 
though, that they are not the same. For example, consider a scenario in which the exponential rate of 
growth of PV production suddenly increased, which would decrease the current production doubling time 
of roughly 3 years. In this case, Wright predicts that the rate at which costs fall would increase, whereas 
Moore predicts that it would be unaffected. Distinguishing between the two hypotheses requires a suffi- 
cient number of examples where production does not increase exponentially, which our current database 
does not contain. The historical data shows a strong tendency, across different types of technologies, 
toward constant exponential growth rates. Recent work has, however, demonstrated super-exponential 



improvement for information technologies 24 over long time spans, suggesting that Moore's law is only 
a reasonable approximation over short spans of time. This evidence from information technology, and 
the results presented here, suggest that Moore may perform significantly worse than Wright over longer 
time horizons. 
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Figure Legends 
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Figure 1. An illustration of the growth of errors with time using the Wright model. The 

mean value of the logarithmic hindcasting error for each dataset is plotted against the hindcasting 
horizon j — i, in years. An error of 10 ' 5 ~ 3, for example, indicates that the predicted value is three 
times as big as the actual value. The longest data-sets are: PrimaryAluminum (green), 
PrimaryMagnesium (dark blue), DRAM (grey), and Transistor (red). 
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Figure 2. An illustration of the growth of errors of each hypothesized law vs. time. The 

plot shows the predicted root absolute log error r/y vs. forecasting horizon (j — i) using each of the 
functional forms (see Eq. ([6|). The performance of the five hypotheses shown is fairly similar, though 
Goddard is worse at short horizons and SKC and Moore are worse at long horizons. 
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Figure 3. Three examples showing the logarithm of price as a function of time in the left 
column and the logarithm of production as a function of time in the right column, based 
on industry-wide data. Wo have chosen these examples to be representative: The top row contains 
an example with one of the worst fits, the second row an example with an intermediate goodness of fit, 
and the third row one of the best examples. The fourth row of the figure shows histograms of R 2 values 
for fitting g and m for the 62 datasets. 
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Figure 4. An illustration that the combination of exponentially increasing production and 
exponentially decreasing cost are equivalent to Wright's law. The value of the Wright 
parameter w is plotted against the prediction m/g based on the Sahal formula, where m is the exponent 
of cost reduction and g the exponent of the increase in cumulative production. 
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Figure 5. An illustration of how individual datasets deviate from the pooled data. The 

data-specific contribution to the slope, bd, is plotted against the data specific contribution to the 
intercept, a<j, and compared to the ellipse of two standard deviation errors. The best forecasts are 
obtained for those found in the lower left quadrant, such as Beer, Sodium, RefinedCaneSugar, and 
Aluminum. 
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Figure 6. A projection of future PV electricity costs from the Photovoltaics2 historical 
data set (1977 2009) using Moore's exponential functional form. The solid line is the 
expected forecast and the dashed line is the expected error. 



